In most of existing deep convolutional neural networks (CNNs) for classification, global average (first-order) pooling (GAP) has become a standard module to summarize activations of the last convolution layer as final representation for prediction. Recent researches show integration of higher-order pooling (HOP) methods clearly improves performance of deep CNNs. However, both GAP and existing HOP methods assume unimodal distributions, which cannot fully capture statistics of convolutional activations, limiting representation ability of deep CNNs, especially for samples with complex contents. To overcome the above limitation, this paper proposes a global Gated Mixture of Second-order Pooling (GM-SOP) method to further improve representation ability of deep CNNs. To this end, we introduce a sparsity-constrained gating mechanism and propose a novel parametric SOP as component of mixture model.

The idea behind a Sparse Gated Mixture (GM) of Expert model has already been proposed in [1]. The main novelty of this paper is in the way the sparse mixture model is applied, namely to modify the 2nd order pooling layer within a deep CNN model to have a bank of candidates. The way GM works is as follow: Given an input sample, the sparsity-constrained gating module adaptively selects Top-K experts from N candidates according to assigned weights and outputs the weighted sum of the outputs of the K selected experts. Another contribution of this paper is to define a parameterized architecture for pooling: For the choice of expert, the authors use a modified learnable version of matrix square-root normalized second-order pooling (SR-SOP) [2] . The experiments first show that SR-SOP is advantageous over regular SOP, both are prior work, but it's good to justify why use SR-SOP in the first place.

In most of existing deep convolutional neural networks (CNNs) for classification, global average (first-order) pooling (GAP) has become a standard module to summarize activations of the last convolution layer as final representation for prediction. Recent researches show integration of higher-order pooling (HOP) methods clearly improves performance of deep CNNs. However, both GAP and existing HOP methods assume unimodal distributions, which cannot fully capture statistics of convolutional activations, limiting representation ability of deep CNNs, especially for samples with complex contents. To overcome the above limitation, this paper proposes a global Gated Mixture of Second-order Pooling (GM-SOP) method to further improve representation ability of deep CNNs. To this end, we introduce a sparsity-constrained gating mechanism and propose a novel parametric SOP as component of mixture model.